Images may be segmented into various regions in order to determine an item of interest represented in the image. In the case of items in a catalog of items, attempting to search for visually similar items in the catalog can be slow and inefficient with large catalogs of items for consumers who are looking to find a specific item quickly. Generally, users can search for items, such as apparel, based on various attributes or categories corresponding to the apparel. Such attributes or categories can include gender, size, and color. Typically, such attributes or categories need to be manually identified and associated with each piece of apparel. In some instances, such information may be provided by the seller of an item. However, in instances where the user wants to locate visually similar items to a particular item of interest, then users may have difficulty locating the item in large catalogs.